Spatio-Temporal Proximity Distribution Kernels for Action Recognition

Spatio-temporal local feature based bag of visual words algorithm (BOVW) has shown promising results in complex human action classification. However, one key disadvantage of BOVW is geometrical unconstraint, which makes it impossible to recognize different actions with the same features but different spatial-temporal distribution of these features. In this paper, we exploit the spatio-temporal proximity distribution of local features in 3D space to characterize geometric context of action class. The obtained spatio-temporal proximity matrix models both the appearance and geometrics of human actions. Moreover, a spatio-temporal proximity distribution kernel (ST-PDK) is proposed to measure the similarity of videos, which satisfies Mercer condition and is directly incorporated into the kernel function of the SVM classifier. Our algorithm achieves the highest classification accuracy on KTH dataset, one of the most challenging and popular action datasets.